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DESIGN  ACTIVITY  IN  THE 
SOFTWARE  COST  REDUCTION  PROJECT 


INTRODUCTION 

This  report  presents  the  results  of  an  investigation  of  design  activities  of  the  software  engineers 
working  on  the  Software  Cost  Reduction  (SCR)  project.  One  purpose  of  this  study  is  to  offer  insights 
into  understanding  the  design  process  of  complex  software.  A  second  purpose  is  to  identify  parameters 
that  characterize  and  predict  design  progress.  The  data  analyses  suggest  that  at  least  one  parameter 
does  characterize  and  predict  design  progress  under  the  SCR  approach. 

The  Software  Cost  Reduction  Project 

Since  1978,  the  Naval  Research  Laboratory  in  cooperation  with  the  Naval  Weapons  Center  has 
been  redeveloping  version  2  of  the  operational  flight  program  for  the  A-7E  aircraft  til.  Software 
engineering  techniques  such  as  formal  requirements  specification,  information  hiding  [2],  abstract  inter¬ 
faces  [31,  and  cooperating  sequential  processes  [4]  are  being  used.  This  effort  is  referred  to  as  the  SCR 
project. 

The  goals  of  the  SCR  project  are  to  (a)  demonstrate  the  feasibility  of  using  selected  software 
engineering  techniques  in  developing  complex,  real-time  software;  and  (b)  provide  a  model  for 
software  design.  The  claimed  advantage  of  the  selected  software  engineering  techniques  is  that  they 
can  facilitate  the  development  of  easy-to-change  software.  Heninger  et  al.  [51  provide  complete  discus¬ 
sion  of  the  project’s  software  requirements.  Britton  and  Parnas  [6l  give  a  detailed  description  of  the 
module  design  structure. 

The  Software  Technology  Evaluation  Project 

The  goal  of  the  Software  Technology  Evaluation  (STE)  project  is  to  evaluate  alternative  software 
development  technologies.*  The  approach  is  to  monitor,  evaluate,  and  compare  software  development 
technologies  used  in  different  software  projects.  The  monitoring  and  evaluating  processes  consist  of 
goal-directed  data  collection  and  analyses  techniques  [71.  One  of  the  tasks  of  the  STE  project  is  to  pro¬ 
vide  the  basis  for  an  objective  evaluation  of  the  methodology  used  in  the  SCR  project.  The  two 
projects  are,  however,  separate  research  investigations  each  with  its  own  goals,  staff,  and  funding. 

DATA  COLLECTION 

Since  1978,  all  SCR  project  engineers  have  been  required  to  submit  weekly  reports  on  their 
project  activity  hours.  The  activity  data  are  collected  on  a  form  called  the  Weekly  Activity  Report,  the 
current  version  of  which  is  presented  in  Chart  1.  The  boxes  on  the  form  represent  different  project 
activities. 

A  submitted  report  is  usually  rather  sparse;  typically,  it  has  only  a  few  boxes  marked  with  hours 
spent  on  project  activities  during  the  week.  A  copy  of  a  completed  report  form  is  presented  in  Chart  2. 
Once  a  weekly  activity  report  is  given  to  STE  project  personnel,  it  is  validated  and  entered  into  a  com¬ 
puter  data  base  An  instruction  sheet  explaining  how  to  report  weekly  activity  is  provided  to  each  SCR 
engineer. 

Manuscript  approved  March  5.  1986 

•This  work  is  currently  funded  by  the  Dol)  STARS  Program  as  Measurement  Area  Task  G-06 
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Module  Development  Activities 

The  front  page  of  the  Weekly  Activity  Report  form  is  primarily  used  to  record  hours  spent  on 
module  development  activity,  where  module  means  information  hiding  module  [2],  As  can  be  seen  in 
Chart  1,  SCR  development  activity  hours  are  captured  for  each  engineer  by  a  specific  module  within 
the  hierarchy  (e  g.,  Device  Interface)  and  by  design,  code,  and  test  categories.  Space  is  provided  for 
project  personnel  to  supply  the  names  of  the  modules  below  the  first  two  levels. 

The  primary  product  of  design  activity  is  the  development  of  a  module  interface  specification.  A 
typical  interface  specification  for  the  Device  Interface  module  (8)  is  presented  in  Chart  3.  Design 
activity  is  reported  as  hours  devoted  to  design  creating,  design  discussing,  design  peer  reviewing,  and 
design  formal  reviewing  activities.  Design  creating  activity  is  time  devoted  to  thinking  about  a  design 
including  redesigning  or  documenting.  Design  discussing  activity  is  time  devoted  to  discussing  design 
issues  via  a  computer  message  or  directly  with  a  colleague  to  assist  with  the  design.  Design  peer  review¬ 
ing  activity  is  time  devoted  to  reading  or  commenting  on  (informally)  design  documentation  produced 
by  another  project  member  in  order  to  assist  with  the  design.  Design  formal  reviewing  activity  is  time 
spent  in  a  formal  design  review,  typically  at  the  Naval  Weapons  Center,  which  maintains  the  current 
operational  flight  program. 

Coding  activity  is  reported  as  hours  devoted  to  pseudocode,  Extended  Computer  code,*  C  code 
and  TC  codet  activities.  Pseudocode  activity  is  further  reported  as  hours  devoted  to  code  creating, 
code  discussing,  and  code  peer  reviewing  activities.  EC  code,  C  code,  and  TC  code  activities  are 
reported  as  hours  devoted  to  code  creating,  code  discussing,  code  peer  reviewing,  and  code  programmer 
testing  activities.  Code  creating,  code  discussing ,  and  code  peer  reviewing  activities  have  definitions  similar 
to  their  counterparts.  Code  programmer  testing  activity  is  time  devoted  by  programmers  to  computer- 
based  evaluation  of  their  own  code  to  convince  themselves  of  its  correctness. 

Test  activity  is  reported  as  hours  devoted  to  test  preparation,  test  conducting,  and  test  reviewing 
results  activities.  Test  preparation  activity  is  time  devoted  to  creating,  discussing,  and  reviewing  plans 
and  procedures  for  computer-based  testing  of  a  module  prior  to  formal  subset  testing.  Test  conducting 
activity  is  time  devoted  to  set  up  and  execution  of  module  test  procedures  on  a  computer.  Test  review¬ 
ing  results  activity  is  time  devoted  to  analyzing,  discussing,  and  documenting  results  of  a  module  test 

Other  Activities 

The  back  page  of  the  weekly  Activity  Report  form  is  used  to  record  hours  spent  on  software  test¬ 
ing  and  miscellaneous  activities.  Software  testing  activity  is  reported  as  hours  spent  on  general  issues  of 
computer-based  testing  and  on  testing  of  system  subsets.  Miscellaneous  activity  is  reported  as  hours 
spent  on  activities  not  included  in  any  of  the  above  definitions. 

OVERVIEW  OF  SCR  PROJECT  ACTIVITIES 

From  January  1978  to  February  1985,  over  55,000  activity  hours  have  been  reported.  Experi¬ 
ments  have  been  performed  to  provide  reasonable  assurance  that  the  reported  hours  accurately  reflect 
project  activity  and  are  appropriately  categorized  (91.  Figure  1  represents  the  monthly  accumulation  of 
hours  expended  on  all  activities.  Figure  2  shows  the  monthly  accumulation  of  hours  expended  in  the 
top  level  categories.  Software  Structures  (SS)  effort  is  time  spent  defining  and  documenting  hierarchi¬ 
cal  module  structure  in  the  A-7E  module  guide  [6],  Software  Modules  (SM)  effort  is  time  devoted  pri¬ 
marily  to  specifying  and  implementing  modules.  Software  Testing  (ST)  effort  is  time  spent  on  valida¬ 
tion  testing  of  subsets,  and  Miscellaneous  (MISC)  is  time  spent  on  all  other  activities  such  as  travel  and 
project  control.  Most  SCR  work  so  far  has  concentrated  on  SM  development. 

*Fhe  Extended  mpuier  is  one  ol  the  modules  ol  the  program  h('  code  is  consists  of  invocations  of  programs  on  this 
module's  interface 

^TC  code  is  the  assembly  language  code  lor  the  IBM  System  4  PI  model  TC-2  computer  The  operational  flight  program  runs  on 
this  machine 
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DI.WOG:  WEIGHT  ON  GEAR  SENSOR 


1.  Introduction 

The  weight  on  gear  device  is  s  sensor  that  detects  whether  or  not  the  aircraft  is  resting  on  its 
landing  gear.  This  data  can  be  used  to  infer  whether  or  not  the  aircraft  is  airborne. 


2.  INTERFACE  OVERVIEW 


2.1.  ACCESS  PROGRAM  TABLE 


Program 


Parameter » 


Deteription 


Undetirei  event* 
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MONTH 

Fig.  1  —  SCR  activity 


MONTH 

Fig.  2  —  SCR  area  activities 

Figure  3  shows  the  monthly  accumulation  of  hours  expended  in  the  Software  Modules  category 
on  design,  code  (including  pseudocode),  and  test  activities.  Over  75%  of  all  reported  software  module 
activity  is  module  design  including  redesign,  ""his  is  consistent  with  the  emphasis  in  the  SCR 
methodology  on  extensive  design  with  the  expectation  of  significant  reductions  in  coding,  testing,  and 
maintenance  efforts  (10], 

There  are  three  categories  of  first-level  SCR  modules:  Hardware  Hiding  modules.  Behavior  Hiding 
modules,  and  Software  Decision  modules  16],  These,  in  turn,  include  ten  categories  of  second-level 
modules,  listed  in  Table  1.  Each  of  the  second-level  modules  is  organized  into  several  submodules 
(third-level  modules),  and  some  of  these  are  further  modularized.  The  EC  module,  with  seven  levels 
ot  submodules,  has  the  deepest  module  structure. 

Six  of  the  second-level  modules  have  accumulated  more  than  1000  hours  of  activity,  these  are 
EC,  Dl,  FD,  SS,  AT,  and  PM.  The  six  also  have  complete  module  interface  specifications  that  are 
baseline  or  nearly  baselined.  In  Figures  4  through  9,  the  monthly  accumulation  of  hours  expended  on 
total  activity  and  on  design,  code,  and  test  activities  is  presented  for  each  module  *  Only  the  EC  and 
DI  modules  have  appreciable  amounts  of  coding  and  testing  activities. 

■Vertical  lines  in  f  igures  4  through  21  represent  the  dates  on  which  baseline  interfaces  specifications  lor  the  respective  modules 
were  released  The  absence  of  these  lines  on  a  specific  plot  indicates  that  no  baseline  documents  have  been  released  for  that 
module 
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Fig  3  —  Software  module  activities 


Table  1  —  Abbreviations  and  Names  of 
Second-Level  Software  Modules 


Abbreviation 

Name 

AT 

Applications  Data  Type 

DB 

Data  Banker 

DI 

Device  Interface 

EC 

Extended  Computer 

FD 

Function  Driver 

FLT 

Filter 

PM 

Physical  Model 

SG 

System  Generation 

SS 

Shared  Services 

SU 

System  Utilities 

Fig  4  —  Extended  computer  activities 
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Fig.  7  —  Shared  services  activities 


Fig  8  —  Applications  data  type  activities 
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Fig.  9  —  Physical  model  activities 


ANALYSES  OF  MODULE  DESIGN  DATA 

As  mentioned  above,  the  SCR  project  emphasizes  careful  design  which  is  reflected  by  the  fact  that 
design  activity  accounts  for  over  75%  of  all  reported  software  module  activity.  One  of  the  purposes  of 
the  data  analyses  was  to  identify  parameters  that  characterize  the  design  processes  and  offer  predictive 
capabilities  concerning  them.  Plots  were  constructed  in  order  to  characterize  monthly  hours  expended 
on  the  subactivities  of  module  design:  design  creating  (DC),  design  discussing  (DD),  design  peer 
reviewing  (DR),  design  formal  reviewing  (DF),  as  well  as  total  design  (D).  Unfortunately,  characteris¬ 
tic  patterns  were  not  readily  apparent. 

Subsequently,  it  was  decided  to  examine  the  accumulation  of  hours  expended  on  total  design 
subactivities.  This  approach  is  considered  appropriate  because  each  data  point  reflects  the  history  of 
design  activities  up  to  that  point  in  time.  Thus,  the  cumulative  total  design  hours  for  a  module  is 
defined  by: 

CumDn  =  £  D,, 

i-l 

where  D,  is  the  monthly  total  of  all  design  activities  on  a  module  for  month  i.  (Note  that  D,  includes 
design  activity  on  all  submodules  of  the  module.)  Because  data  are  available  for  all  the  months 
between  January  1978  and  February  1985,  n  has  values  from  1  to  86.  Cumulative  design  creating  hours 
for  a  module  is  defined  by: 

CumDCn  =  £  DC, , 

i- 1 

where  DC,  is  the  monthly  design  creating  subactivity  for  a  module  (including  all  submodules)  for 
month  i.  Again,  n  has  values  from  1  to  86.  Similar  definitions  apply  for  CumDDn  and  CumDRn.  Fig¬ 
ures  10  through  15  show  the  significant  cumulative  design  activities  for  each  module. 

An  earlier  study  19)  highlighted  the  fact  that  ratios  between  activity  categories  provide  valid  and 
potentially  useful  metrics  of  SCR  project  activity.  STE  Project  personnel  intuitively  suspected  that 
ratios  between  activity  categories  could  provide  descriptive  features  of  the  SCR  methodology  that  might 
be  generally  applicable  to  software  design.  Consequently,  six  ratio  series  between 
CumDCn,  CumDDn,  and  CumDRn  were  computed.  For  example,  the  ratio  between  cumulative  design 
discussing  and  cumulative  design  creating  is  defined  as: 
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(CumDD/CumDC)n  = 


CumDDn 

CumDC„ 


where  n  has  values  from  1  to  86.  The  other  five  ratios,  similarly  defined,  are 
(CumDC/CumDD)n,  (CumDC/CumDR)n,  (CumDD/CumDR)n,  (CumDR/CumDC)n,  and  (CumDR/ 
CumDD)n.  These  ratios  were  correlated  with  CumD„  over  the  86  reporting  months.  Pearson  correla¬ 
tion  coefficients  [II]  are  presented  in  Table  2.  Next,  for  each  module  the  ratio  between  monthly  DC, 
DD,  and  DR  were  computed  (e  g.  DCn/DD„,  DCn/DRn,  and  so  on)  and  correlated  with  the  total 
monthly  Ds  and  with  the  monthly  CumDs.  These  coefficients  are  presented  in  Tables  3  and  4. 
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Fig  10  —  Extended  computer  design  activities 
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Fig  II  —  Device  interface  design  activities 
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Fig.  14  —  Applications  data  type  design  activities 
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Fig  15  —  Physical  model  design  activities 


Table  2  —  Pearson  Correlation  Coefficients  Between  CumD  and  Cum.  Ratios 


Module 


CumDC 


CumDD 


.4034 

,0560a 

.3785 

.7565 

.4090 


CumDC 


CumDR 


0. 1 245a 
— 0. 1 998a 
— 0.309 1 
-0.0897“ 
-0.4144 


CumDD 


CumDC 


0.9774 

0.6574 

0.9492 

0.9482 

0.9052 


CumDD 


CumDR 


.9609 
.1002“ 
.6530 
.0252“ 
0.1429“ 


BSC 

I 


CumDR 


CumDC 


.7483 
.8324 
.5436 
0.1001“ 
0.9235 


CumDR 


CumDD 


-0.2006“ 

0.8124 

0.5436 

0.9575 

0.3801 


.9181  0.8665  0.5156  0.8408  -0.3675  -0.0568 
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Table  3  —  Pearson  Correlation  Coefficients  Between  D  and  Monthly  Ratios 


Module 

DC/DD 

DC/DR 

DD/DC 

DD/DR 

DR/DC 

DR/DD 

AT 

0.3099 

0.4031 

0  3548 

0.3048 

0.3664 

0. 1 572a 

DI 

0.3759 

0.4829 

0.1040“ 

0.2044a 

0.02273 

0.20753 

EC 

-0.0008“ 

0.3972 

0.3401 

0.5396 

0,0040a 

-0.0998“ 

FD 

0.6470 

0.5190 

0.13053 

0.4813 

0.10423 

0.3796 

PM 

0.3083 

0.5845 

0.3175 

0.6581 

0.046  la 

0.3491 

SS 

0.6509 

0.6040 

0.046  la 

0.4309 

0.1248“ 

0.2578 

dNot  significant  at  the  005  level 


Table  4  —  Pearson  Correlation  Coefficients  Between  CumD  and  Monthly  Ratios 


Module 

DC/DD 

DC/DR 

DD/DC 

DD/DR 

DR/DC 

DR/DD 

AT 

0.0532“ 

0.16673 

0.11693 

0.08433 

-0.01773 

-0.1287“ 

DI 

-0.1693“ 

-0.1414“ 

0.03443 

-0.08793 

0.06833 

0.0352“ 

EC 

-0.0932“ 

0.3933 

0.3467 

0.5236 

—0.0248“ 

-0.0886* 

FD 

-0.0034“ 

0.0603  3 

0.14093 

0.0293“ 

0.1032“ 

0.1282“ 

PM 

0.4034 

0.3568 

0.11823 

0.2205 

0.170“ 

0.2582 

SS 

0.05473 

-0.04473 

0.15563 

— 0.053 1 a 

-0.0177“ 

0.0815“ 

dNot  significant  at  the  005  level. 

RESULTS 

An  examination  of  the  correlation  coefficients  reveals  that  the  ratio  (CumDD/CumDC)n  corre¬ 
lates  consistently  well  with  CumDn,  as  shown  in  Table  2*  This  relationship  is  evident  from  the  plots 
of  the  ratios.  In  Fig.  16,  the  monthly  ratio  for  (CumDD/CumDC)„  is  plotted  Tor  the  EC  module. 
Comparing  this  with  Fig.  10,  one  can  see  that  design  activity  surges  are  characterized  by  prior  or  con¬ 
comitant  dramatic  increases  in  this  ratio.  When  this  ratio  remains  constant,  it  is  an  indication  that 
design  activity  has  stabilized.  Increases  in  this  ratio  seem  to  indicate  progress.  Consequently,  we  refer 
to  th;s  as  the  progress  indicator  ratio  (PIR) 

Although  the  EC  module  is  extremely  large  and  complex,  the  relationship  seems  strong.  A  large 
jump  in  design  activity  follows  the  large  rise  of  the  PIR.  However,  design  activity  for  this  module  is 
not  quite  stabilized,  and  the  late  downward  trend  in  the  ratio  indicates  increasing  creating  time  relative 
to  discussing  time 

The  same  patterns  are  also  present  for  the  D1  module  As  can  be  seen  in  Figs.  12  and  17,  the 
dramatic  increase  in  design  activity  follows  a  dramatic  increase  in  the  progress  indicator  ratio.  This 
same  relationship  holds  for  the  FD,  SS,  AT,  and  PM  modules.  See  Figs.  18  through  21. 

Coefficients  of  determination  (r:),  as  defined  in  Ref.  11,  between  C'umDn  and 
(CumDD/CumDC)n  are  presented  in  Table  5  for  each  module.  This  ratio  seems  to  explain  a  high  per¬ 
centage  of  the  variation  of  CumD„. 

The  analyses  provide  supporting  evidence  that  the  ratio  (CumDl)/CumDC)n  is  an  important 
measure  of  design  activity  progress  in  developing  modules  for  complex  software.  When  the  PIR 
becomes  constant,  design  activity  appears  to  be  at  a  very  low  level  or  even  nonexistent.  When  this 
ratio  increases,  design  activity  increases  dramatically.  The  relationship  between  this  ratio  and  CumDn  is 
the  strongest  of  all  the  possible  relationships  examined  in  this  study  In  at  least  one  module,  this  ratio 
can  explain  over  95%  of  the  variation  in  CumDn.  In  the  remaining  modules,  variations  in 
(CumDD/CumDC'n  can  explain  a  surprisingly  high  degree  of  the  variations  in  CumDn 

*The  probability  of  finding  this  significant  result  is  not  increased  because  the  same  analyses  were  conducted  over  different  data 
sets 
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Jan  78  Jan  79  Jan  80  Jan  81  Jan  82  Jan  83  Jan  84  Jan  85 

MCNTH 


NORCIO  AND  CHMURA 


NORCIO  AND  CHMURA 


CONCLUSIONS 

A  natural  conclusion  is  that  discussion  between  software  designers  is  a  critically  important  factor 
in  the  design  of  information-hiding  modules  for  complex  software.  When  the  release  dates  for  specifi¬ 
cation  baseline  (e  g.,  Ref.  8)  are  examined  with  the  P1R,  the  P1R  seems  to  be  indicating  the  complete¬ 
ness  of  the  baseline  specifications  When  a  baseline  appears  bet  ore  this  ratio  rises  sharply  or  during  a 
sharp  rise,  the  baseline  is  probably  far  from  complete.  Abstract  interface  specifications  would  seem  to 
become  reasonably  stable  only  after  a  sharp  rise  and  settling  of  this  ratio.  Plotting  this  ratio  over  time 
may  provide  for  the  software  manager  a  meaningful  tool  with  which  to  track  design  progress.  If  the 
PIR  has  not  surged  and  stabilized,  the  design  is  probably  not  finished  irrespective  of  personnel  claims 
and  published  baseline  documents. 

In  addition,  the  PIR  has  an  attractive  property  not  found  in  a  monthly  plot  of  CumDn.  The  range 
of  the  y-axis  is  constant  over  time  and  over  other  modules  and  projects.  Therefore,  it  is  possible  to 
compare  design  progress  on  one  module  or  project  to  another  by  using  this  ratio.  The  PIR  does,  how¬ 
ever,  have  one  possible  negative  property.  Because  it  involves  cumulative  sums,  the  accumulation  of 
earlier  design  hours  can  dampen  the  impact  of  later  variations  in  design  activity.  The  PIR  for  the  EC 
module  indicates,  however,  that  this  possible  flaw  may  be  more  theoretical  than  practical. 

There  is  no  claim  that  the  PIR  is  a  measure  of  design  completeness.  There  are  clearly  other  rea¬ 
sons  why  design  activity  on  a  specific  software  module  may  have  stabilized;  for  example,  personnel  may 
have  shifted  work  to  another  module  or  they  may  have  been  vacation.  However,  the  PIR  ratio  seems 
to  indicate  when  work  on  a  piece  of  software  is  definitely  not  finished.  If  design  completion  is  claimed 
prior  to  a  rise  and  settling  in  this  ratio,  there  is  probably  more  work  that  needs  to  be  done  on  that 
module 

It  is  necessary  that  this  analysis  be  replicated  on  other  large  scale  software  development  projects  to 
determine  whether  the  PIR  behaves  similarly  in  other  software  development  environments  using  dif¬ 
ferent  methodologies.  It  is  intuitively  appealing  that  discussion  between  project  members  necessarily 
enhances  the  design  of  software  modules.  It  would  also  be  useful  to  quantify  the  relative  surges  in  the 
PIR.  That  is,  there  is  practical  importance  in  knowing  that  a  given  percentage  increase  in  the  PIR  is 
customarily  followed  by  a  predictable  percentage  increase  in  design  activity.  This,  too,  requires  replicat¬ 
ing  these  analyses  in  several  different  software  design  environments.  Unfortunately,  these  data  are  dif¬ 
ficult  to  collect  and  it  is,  perhaps,  even  more  difficult  to  validate  their  accuracy. 

Finally,  it  is  logical  to  examine  coding  data  for  these  relationships.  It  seems  reasonable  to  accept 
the  importance  of  discussion  in  the  design  process.  Its  importance  in  the  coding  and  testing  processes 
is  not  as  clear.  These  data  do  exist  in  the  SCR  data  base  and  plans  are  under  way  to  examine  them. 
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